Abstract: What do models know and how? Answering this question requires exploratory analyses comparing many models, but existing techniques are specialized to specific models and analyses. We consider visualizing the model's sensitivity to its training data. Our approach, Memory Maps, is an extension of residual-leverage diagnostic plots where the two criteria are derived for a wide range of models and algorithms by using a Bayesian framework. The modified criteria are used to understand a model's memory through a 2D scatter plot where tail regions contain examples with high prediction-error and variance. All sorts of models can be analyzed this way, including not only those arising in kernel methods, Bayesian methods, and deep learning but also the ones obtained during training. We show use cases of Memory Maps to diagnose overfitting, compare various models, and analyze training trajectories.